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iBSTBlCT 

Telephone interviews were conducted to compare 
reported and trae date of birth and salary for 105 Hezican Asericant 
219 Anglo American, and 134 black city employees in San Antonio, 
Texas. The Anglo and Hezican Aierican samples vere ran dofly selected 
f roB the police and fire departaents; the blacks, not being 
concentrated in sofficient nuabers in any few departments, were t>ke& 
froa all 22 city departBcnts. Telephone nuabers and data about- sei> 
birth, and eaployaent dates, and salary were obtained; froi a doc^^ 
filled oat by employees in the Personnel Depart ient^^i gaest^^ 
vas written to guide in eliciting responses about the saae fa 
inforuatxon taken fros the p^i^sonnel records. Two Anglo f euales who 
were experienced in interviewing for scholarly and coupercial 
research conducted the interviews. Significance of differences tests 
were conducted with the chi-square procedure. Inaccurate responses 
were categorized into ower-^ and under statement • Soie findings were: 
(1) of the 455 responses to the birth date question^ 20 of the An^lo, 
9 of the Hezican Aierican, and 21 of the black responses were 
inaccurate; (2) of the 389 responises'to the salary question, only. 33 
were correct within one dollar; and (3) the younger, but less 
well-paid blacks tended to report their salaries uore accurately. 
(HQ) 
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A COMPARATIVE STUDY OF THE VALIDITY 
OF INTERVIEW RESPONSES BY 
MEXICAN-AMERICANS, ANGLO-AMERICANS, AND BLACKS 



The problem of validity in survey research arises because respon- 

1 

dents do not alv/ays tell the truth* Even the most carefully planned 
research project is destined to frilure if respondents £,^o3tantially mis- 
represent facts to interviewers. Two schools of thought exist with respect 

2 

to the definition of validity. Social psychologists, educators and others 

involved with psychological testing are concerned with validity in terms of 

predictive accuracy. Thus, the validity of attitude and opinion surveys 

is measured by their ability to predict the performance of individuals in 

3 

some future situation. Many social researchers have come to realize, 

however, that confining the definition of validity to predictive accuracy 

4 

is an unwarranted limitation on the use of survey results. Opinion may 
be closely related to validity and yet have a unique validity of its own. 
In dealing with the validity of factual information, a new definition of 
validity was necessary. Hence, the concept of validity has also come to 

^Carol H. Weiss, "Validity of Welfare Mothers* Interview 
Responses, " Public Opinion Quarterly, XXXII (Winter, 1968-1969), 622. 

2 

Hugh J. Parry and Helen M. Crossley, "Validity of Responses 
to Survey Questions, " Public Opinion Quarterly, XIV (Spring, 1950), 61-64. 

^ Ibid . 

^Ibid* 
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mean the degree of accuracy with which respondents reply to survey 
5 

questions. This investigation is concerned with the concept of truth- 
fulness as a determinant of validity. 

A prerequisite of validity is reliability which refers to the con- 

6 

sistency of responses elicited through surveys and tests. Reliability 

is measured by means of a test-retest procedure with the objective of 

determining whether the same interviewee will respond differently to 

7 

the same question at different times. Consequently a survey must be 
reliable in order to be valid since it must always elicit an accurate 
response. A reliable survey could consistently evoke the same answer, 
however, and still be invalid since no consideration would be given to 
the accuracy of response. In order to be valid, a survey must accurately 
measure and report the data for which it was designed to measure. 

Because of the difficulty involved in verifying answers, even when 
respondents can be identified, relatively little work has been done in this 

g 

area. An even lesser explored facet of validity research is the error 
involved in survey responses by members of the two largest minority 

^ rbid. 

^LeeJ. Cronbach, Essentials of Psychological Testing (New York; 
Harper and Brothers Publishers, 1960), p. 126. 

^Frank S. Freeman, Theory and Practice of Psychological Testing 
(Chicago: Holt, Rinehart and Winston, 1962), p. 66. 

^Cronbach, loc. cit. 

"Parry and Crossley, op. cit. , p. 64. 
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groups in the United States, Mexican-Americans and Blacks. The authors 
are aware of only one study which was designed s^pecifically to determiiie 
the validity of the responses of minority group members to interviews, 
and this study dealt only with the validity of reported welfare payments by 
Black welfare mothers. The lack of knowledge in this area is partially 
attributable to the difficulty involved in gaining access to^ factual informa- 
tion on a controllable concentration of minority group members who may 
be contacted for interviews. Moreover, the difficulty involved in locating 

and contacting certain /niiiority group members is often considerably 

11 

greater than that involved in reaching the white Anglo respondent. 

METHOD 

We conducted telephone interviews with 219 Anglo-American, 105 
Mexican- American, and 134 Black employees of the city of San Antonio. 
The Anglos and Mexican- Americans were randomly selected from the 
police and fire departments, but the Blacks, not being concentrated in 
sufficient numbers in any few departments, were taken from all of the 

^^Weiss, "Validity of Welfare Mothers'. Interview Responses, " 

p. 622. 

^^Weiss, ^'Interviewing Lx>w- Income Respondents, Welfare in 
Review, IV (October, 1966), 2-3. 
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city's twenty-two departments. The Blacks who were interviewed were 
all those employed by the city who could be reached by telephone. 

Telephone numbers and information about sex, date of birth, date 
of employment, and salary were obtained from a document in the records 
of the Personnel Department which was filled out by employees. (They 
sign a statement certifying that they understand that the personnel office 
will investigate the accuracy of their answers and that providing misin- 
formation is grounds for dismissal.) A questionnaire was written to guide 
in the eliciting of responses about the same factual information taken from 
*he personnel records. Our questions were not framed in the interest of* 
securing the most accurate data, but to determine the s^ccuracy of re- 
sponses to simple, straightforward questions. The interview was 
conducted as follows: 

Hello. Is Mr. at home? Mr. . my name is Carol 

Hall. I am conducting a survey on voting for Professor Weaver 
ot St. Mary*s University. Do you think 18-year olds should be 
allowed to vote? When were you bom? Do you have a full-time 
job? When did you start working there? What is ^our monthly/ 
hourly salary? Thank you for your cooperation. Goodbye. 



"The overall results— not broken down by ethnicity— are reported 
elsewhere. See Charles N. Weaver and Carol L. Swanson "Validity of 
Reported Date of Birth, Salary, and Seniority, " The Public Opinion 
Quarterly, Vol. 38 (1974), 69-80, and Charles N. Weaver, Sandra L. 
Holmes, and Norval D. Glenn, "Some Characteristics of fiiaccessible 
Respondents in a Telephone Survey, " Journal of Applied Psychology, 
Vol. 60 (1975), forthcoming. 

**For giving us access to personnel data, we are indebted to Mr. 
Clyde C. McCollough, Jr. , Director of Personnel, City of San Antonio. 
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The necessity for differentiating between monthly and hourly 
wage earners on the questionnaire was created by the fact that many 
Black employees are paid an hourly wage and worked a variable number 
of hours each month. Consequently, their hourly wage could not be con- 
verted to a monthly figure. 

The interviews were conducted by two Anglo females, one of 
whom has a Master^s degree in English and the other was a graduate 
student in Business Administration. Both were experienced in inter- 
viewing for scholarly and commercial research. They were familiar 
with and sought to avoid the response errors which are thought to result 

0 

from disparity in social class between the two parties to an interview. " 

The two interviewers did not use their correct names because 
it was preferred that inquiries be directed to their supervisor at St. Mary^s 
University rather than to the interviewers themselves, fii previous research 
we discovered that associating a survey with a university had a favorable 
effect on the response rate since respondents seem to be more receptive 
to a study sponsored by an educational institution than to a commercially 
sponsored survey. The lead question "Do you think 18 year olds should 
be allowed to vote?" was chosen because a pilot test revealed that many 
of the interviewees felt a social responsibility to express their opinions 

'""For a discussion of such errors, see 

Daniel Katz, "Do Interviewers Bias Polls?" Public Opinion Quarterly , 
VI (Summer, 1942), 248-268, and James A. Williams, Jr., "Interviewer- 
Respondent Interaction: A Study of Bias in the Information. Interview, " 
Sociometry. XXVII (September, 1964). 338-352. 
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on a matter of perceived national concern. Another important reason 
for using the voting question was that respondents seemed to expect 
that we would ask them about their age and other control information 
in order to evaluate their response. Thus, this question provided a 
successful ingress to the rather personal questions which followed* 

No reference was made to the location of the respondent's em- 
ployment since it was necessary for the purpose of the study to avoid 
any suggestion that responses could be or would be checked for truthful- 
ness. We did, however, preface the salary question with an inquiry 
about whether the respondent had a full-time job so that anyone who v/as 
moonlighting would know that our subsequent questions referred to his 
employment with the city of San Antonio. We followed the salary qu^^- 
tion with an inquiry about occupation. If the answer did not indicate that 
the respondent was referring to his employment at the city, the inter- 
viewer sought clarification to the earlier question about the full-time 
Job. If any respondent had persisted in stating that he was in an occu- 
pation which showed that he was not a full-time city employee,we would 
have inferred that he had terminated his employment after our lists of 
respondents were obtained. Less than two percent of the respondents 
fell into this category. 

There were several reasons for not attempting to measure the 
validity of other traditional controls. For certain variables, such as 
marital status and number of children, it was difficult to secure up-to- 
date information because the Person»^el Office has no administrative 



reason to keep it current. We hesitated to ask additional personal 
questions for fear of increasing the number of complaints or the number 
of terminations. We were able to separate the city employees by race 
on the basis of information from the Personm^l Department, and we 
divided the non- Blacks into Mexican- American and Anglo-American 
on the basis of Spanish-surname. 



FINDINGS 

Reported Date of Birth . 

Of 455 responses to the date of birth question (219 from Anglos, • 

105 from Mexican-Americans, and 131 from Blacks) 50 were inacc|jratei 

1 

A response to this question was considered inaccurate if it was on&^or 
more years over or under the respondent's correct date of birth. There 
was a statistically significant difference (p4.10) among the three groups: 



*While errors in classification, associated with such circum- 
stances as exogenous marriages, can occur by using Spanish-surname 
to identify Mexican -Americans, it is sufficiently accurate to be used by 
most researchers who study Mexican-Americans. For instance, see 
Weaver, Charles N. and Sandra L. Holmes, "Education As an Explanatio 
for the Differential Employment Rate of Mexican-Americans and Anglo- 
American Employees, " Southern Journal of Educational itesearch. Vol. 6 
(1 972); Weaver, Charles N. and Norval D. Glenn, "The Job Performance 
of Mexican Americans, " Sociology and Social Research, Vol. 54 (1970); 
and Weaver, Charles N. "Accidents As a Measure of the Acclimation 
of Mexican Americans, " The Sociological Quarterly, Vol. 11 (1970). 
As w-? use the terms, both Mexican-Americans and Anglos are white 
persons, but the Anglos have no Mexican or colonial Spanish ancestry. 
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20 (9. .13270) of the Anglo responses were in en or, 9 (8. 5714%) of the 
Mexican- American responses were in error, and 21 (16.00%) of the 
Black responses were in error. " Significance tests of pair-wise 
differences among the three grouos showed that the source of the over- 
all variations was attributable to the greater error in the responses by 
Blacks. 

Arrangement of the inaccurate responses into categories of 
over- and understatement reveals a greater tendency for Mexican- 
Americans and Blacks to say that they were younger than they actually 
were: only 7 (3S. 84%) of the 19 inaccurate Anglo responses were under- 
statements but 6 (66. 66%) of the 9 inaccurate Mexican- American 
responses and 14 (66. 66%) of the 21 inaccurate Black responses were 
understatements.** Analysis of the pattern of inaccurate responses 
by our controls (age, salary, seniority, and job classification) revealed 
a common tendency toward more overstatement by younger, less well- 
paid employees, but showed no statistically significant variation among 

the three groups. 

perhaps the most important result of this analysis was whether 
a difference would result from the analysis of a dependent variable with 
reported date of birth instead of true date of birth. We tabled the "yes" 



^Throughout this paper tests for the significance of differences 
are conducted with the chi -square procedure. 



P<. 02. 
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responses to the voting-preference question by both reported and true 
dates of birth. Coinparing the dependent variable arranged in these 
two ways showed no significant difference for any of the three ethnic 
groups.* We concluded that the inaccuracies we had observed in re- 
ported date of birth were not sufficiently large for any of the three 
groups to produce significant distortions in the analysis of a dependent 
variable. 
Reported Salary . 

Of 389 responses to the salary question (186 from Anglos, 92 
from Mexican-Americans and 111 from Blacks) only 33 answers were 
correct within one dollar. Surpr-.singly, 32 of the correct responses 
were from Blacks while only one was from Anglos and none was from 
Mexican-Americans. Among the 111 Black respondents, 86 were paid 
on a monthly basis and 25 were paid by the hour. A greater proportion 
of correct responses was paid on an hourly basis than from those paid 
on a monthly basis. Of the 25 Blacks paid hourly, 9 (36. 00%) gave cor- 
rect salary answers and of the 86 paid by the month, 23 (26. 75%) gave 

correct answers. 

We analyzed the differences among the three groups with respect 
to the direction of their inaccurate responses. If a respondent said he 
earned more than he actually did (by an amount in excess of one dollar), 
he was considered to have overstated his salary; if he reported his salary 
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*With three degrees of freedom each, chi-square analysis re- 
vealed the following results: Anglos, .50<P<. 70; Mexican-Americans, 
^ p>. 95 and Blacks, P>. 90. qqj^^ 



to be less than it actually was (in excess of a dollar),he understated 
it. These comparisons revealed pronounced differences: 164 (88.79%) 
of 186 Anglo responses were overstatements, 71 (77.17%) of 92 
Mexican- American responses were overstatements, and 32 (37. 21%) . 
of the monthly and 9 (36. 00%) of the hourly Black responses were 
overstatements. Conversely. 21 (1 1 . 23%) of Anglo, 21 (22. 83%) of 
Mexican- American, and 31 (36.05%) of monthly and 9 (36.00%) of 
hourly Black responses were understatements. 

We further analyzed inaccurate responses by dividing the over- 
and understatements by degree of inaccuracy: within $25, from $25 
to $50, and over $50. Because of the small cell sizes which resulted, 
we could not perform this analysis with 16 incorrect responses from 
Blacks who were paid by the hour. 

When overstating salary, 25 (78. 12%) of the 32 overstating 
Anglos were within $25 of the true figure while the corresponding 
numbers were 7 (9. 86%) for 71 overstating Mexican -Americans and 

12 (37. 50%) for 32 overstating Blacks. Overstatements which were 
within $25 to $50 of the true figure numbered 60 (36. 59%) for Anglos. 
24 (33. 80%) for Mexican- Americans, and 7 (20. 88%) for Blacks. Re- 
sponses which were inaccurate by more than $50 were given by 79 
(48. 17%) of the Anglos, 40 (56. 34%) of the Mexican- Americans, and 

13 (40.62%) of the Blacks. 
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Arranging the understated responses by the same categories of 
inaccuracy revealed the following variations. In the category of within 
$25. there were 11 (52.38%) of tb- :i underreporting Anglos, 10 (47.62%) 
of the 21 underreporting Mexican .nmericans, and 13 (41. 94%) of the 31 
underreporting Blacks. In the $25 to $50 category there were 4 (19.05%) 
Anglos, 8 (38.10%) Mexican-Americans, and 8 (25.81%) Blacks. Under- 
stating by $50 or more were 6 (28. 57%) Anglos, 3 (14. 28%) Mexican- ^ 
Americans, and 10 (32.25%) Blacks. 

Next, we analyzed the inaccurate responses with our controls. 
The first approach was to examine inaccuracy without regard to whether 
it represented an over- or understatemenl. Since there was only one 
response which was accurate to the dollar among the Anglos and Mexican- 
Americans, this analysis could only be done with responses from Blacks. 
There was no significant variation in the accuracy of Black responses 
with respect to sex or seniority, but there were significant systematic 
variations with regard *o age and salary. The younger, but less well- 
paid Blacks tended to report their salaries more accurately. 

We then applied our controls to the inaccurate responses with 
respect to whether they were over- and understatements. Separating 
the over- and understatements of the three groups by seniority revealed 
no variation except that the Anglos with less seniority tended to overstate 
more than did Anglos with more seniority. The only variation which 
resulted from the separation by age was the general tendency for younger 
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employees in each group to overstate salary. And there was a tendency^ 
common to the three groups toward the overstatement of salary by 
smaller amounts as the amount of true salary was greater. 

The controls were next applied to degrees of over- and under- 
statement. Unfortunately the number of Blacks in the sample was too 
small to permit meaningful analysis of these detailed cross-references. 
Comparison of the Anglos and Mexican-Americans, however, revealed 
several interesting variations. When overstating salary Mexican- 
Americans tended to overstate by relatively more than did overstating 
Anglos. This continued to be true when age, seniority, salary, and job 
classification were controlled. Too few cells were large enough to per- 
mit a similar analysis for understatements. 

In order to determine if important distortions would result from 
the analysis of a dependent variable using i -ported and in contrast to 
true salary, and if any such variation could be attributed to ethnicity, 
we tabled the *'yes" responses to the voting-preference question by re- 
ported and true salaries. Analysis of these data, presented in Tables 1 
and 2, revealed that statistically significant distortions did occur in the 
analysis of a dependent variable using reported salary instead of true 
salary for Anglos and for Mexican- Americans but not for Blacks, 
whether they were paid on an hourly or monthly basis. 
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CONCLUSIONS 

The findings of this research comparing reported and true date of 
birth and salary for Mexican-American, Anglo, and Black employees of 
the city of San Antonio reveal wide variation between the two variables in 
the degree to which they are accurately reported and in the degree of accu- 
racy with which the two variables are reported by the three groups. We 
foimd that salary variations are so significant— among Anglos and 
Mexican- Americans — that great care should be exercised in the use of 
this variable gathered in this way. We believe tb'^t these results raise 
serious doubt that validity is a generalized phenomenon; it is probably 
specialized, being structurally associated with the characteristics of 
respondents. 

It is our impression that the interview responses of Mexican- 
Americans can be expected to be largely similar to those of the larger 
population, but our evidence suggests that Black response behavior is 
sufficiently unique (more accurate in some instances) to require that 
it continue to be the subject of investigation. 
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Table 2 



A Comparison of Reported and Time Salary of 
Black Respondents Answering Yes on tlie Voting 
Preference Question.* 



Salary 

$2.15 per hour or less 
More than $2.15 per hour 

Total 

Less than $400 
$400 - $499 
$500 - $599 
$600 and Over 

Total 



Reported 



True 



N 


% ■ 


N 


% 


11 


57.89 


12 


63.16 


_8 


42.11 


_2 


36.84 


19 


100.00 


19 


100.00 


25 


35.71 


27 


38.58 


16 


22.86 


18 


25.71 


14 


20.00 


13 


18.57 


15 


21.43 


12 


17.14 


70 


100.00 


70 


100.00 



*For the hourly employees, P^.907 for the monthly 
employees P<.90. 
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